A video includes a plurality of pictures, and each of the pictures includes a predetermined number of pixels. The video is encoded on a per picture basis, and each picture is encoded on a basis of a block obtained by dividing the picture.
In general video coding, the amount of information is compressed by reducing redundancy in temporal and spatial directions.
In inter-picture predictive coding for reducing temporal redundancy, prediction information is generated by performing motion estimation and motion compensation on a current picture to be encoded with reference to a picture temporally preceding or succeeding the current picture to be encoded, and a difference between the prediction information and the current picture is encoded. Here, the picture temporally preceding the current picture to be encoded is a picture (forward picture) having a display time earlier than that of the current picture to be encoded, and the picture temporally preceding the current picture to be encoded is the picture (backward picture) having a display time later than that of the current picture to be encoded.
In the MPEG-4 AVC/H.264 method (hereinafter referred to as the H.264 method) defined in the ISO/IEC 14496-10 Advanced Video Coding (AVC) that is one of image encoding methods (video encoding methods), it is possible to perform motion compensation on a current picture to be encoded with reference to arbitrary two pictures temporally preceding or succeeding the current picture to be encoded that is a target to be encoded (for example, see Non-patent Literature 1). For this reason, when inter-picture predictive encoding and inter-picture predictive decoding are performed in the H.264 method, there is a need to store all of the forward and backward pictures which may be referred to (hereinafter referred to as candidate reference pictures).